The use of image data in feedback control of a mechanical system, commonly referred to a visual servo control, is an established and diverse field. There are many approaches, including image-based visual servoing (IBVS), position-based visual servoing, partitioned methods, and switching methods. Most visual servo control methods have a common goal, such as a goal image, that is used to regulate a camera, robot manipulator, or unmanned vehicle to a desired camera pose or trajectory. For example, an image of a scene may be captured (the goal image) and later images may be captured and compared with the original image after the camera has been moved to identify when the camera has returned to its original position (the goal pose).
In many vision-based control tasks, there is no goal image from which to work, as well as no goal pose or trajectory. For instance, if the desired task is to maintain multiple moving objects in the camera filed-of-view, no goal image exits because the motion and position of the objects are not known ahead of time. In such situations, visual servo control methods may be ineffective.